On the Undecidability of Probabilistic Planning and In nite-Horizon Partially Observable Markov Decision Problems

We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exists a plan with success probability exceeding a desirable threshold. Analogous policy-existence problems for partially observable innnite-horizon Markov decision processes under discounted and undiscounted total reward models , average-reward models, and state-avoidance models are all shown to be undecidable. The results apply to corresponding approximation problems as well.